Region-Based Image Retrieval Revisited

Region-based image retrieval (RBIR) technique is revisited. In early attempts at RBIR in the late 90s, researchers found many ways to specify region-based queries and spatial relationships; however, the way to characterize the regions, such as by using color histograms, were very poor at that time. Here, we revisit RBIR by incorporating semantic specification of objects and intuitive specification of spatial relationships. Our contributions are the following. First, to support multiple aspects of semantic object specification (category, instance, and attribute), we propose a multitask CNN feature that allows us to use deep learning technique and to jointly handle multi-aspect object specification. Second, to help users specify spatial relationships among objects in an intuitive way, we propose recommendation techniques of spatial relationships. In particular, by mining the search results, a system can recommend feasible spatial relationships among the objects. The system also can recommend likely spatial relationships by assigned object category names based on language prior. Moreover, object-level inverted indexing supports very fast shortlist generation, and re-ranking based on spatial constraints provides users with instant RBIR experiences.Comment: To appear in ACM Multimedia 2017 (Oral

Towards Fully Automated Manga Translation

We tackle the problem of machine translation of manga, Japanese comics. Manga translation involves two important problems in machine translation: context-aware and multimodal translation. Since text and images are mixed up in an unstructured fashion in Manga, obtaining context from the image is essential for manga translation. However, it is still an open problem how to extract context from image and integrate into MT models. In addition, corpus and benchmarks to train and evaluate such model is currently unavailable. In this paper, we make the following four contributions that establishes the foundation of manga translation research. First, we propose multimodal context-aware translation framework. We are the first to incorporate context information obtained from manga image. It enables us to translate texts in speech bubbles that cannot be translated without using context information (e.g., texts in other speech bubbles, gender of speakers, etc.). Second, for training the model, we propose the approach to automatic corpus construction from pairs of original manga and their translations, by which large parallel corpus can be constructed without any manual labeling. Third, we created a new benchmark to evaluate manga translation. Finally, on top of our proposed methods, we devised a first compleheisive system for fully automated manga translation

Endogenous CCL21-Ser deficiency reduces B16–F10 melanoma growth by enhanced antitumor immunity

The chemokine CCL21 regulates immune and cancer cell migration through its receptor CCR7. The Ccl21a gene encodes the isoform CCL21-Ser, predominantly expressed in the thymic medulla and the secondary lymphoid tissues. This study examined the roles of CCL21-Ser in the antitumor immune response in Ccl21a-knockout (KO) mice. The Ccl21a-KO mice showed significantly decreased growth of B16–F10 and YUMM1.7 melanomas and increased growth of MC38 colon cancer, despite no significant difference in LLC lung cancer and EO771 breast cancer. The B16–F10 tumor in Ccl21a-KO mice showed melanoma-specific activated CD8+ T cell and NK cell infiltration and higher Treg counts than wild-type mice. B16–F10 tumors in Ccl21a-KO mice showed a reduction in the positive correlation between the ratio of regulatory T cells (Tregs) to activated CD8+ T cells and tumor weight. In Ccl21a-KO tumor, the intratumoral Tregs showed lower co-inhibitory receptors TIM-3 and TIGIT. Taken together, these results suggest that endogenous CCL21-Ser supports melanoma growth in vivo by maintaining Treg function and suppressing antitumor immunity by CD8+ T cells